Data centers may host applications and store large amounts of data for an organization or multiple organizations. Computer networks in these data centers rely on sophisticated services to enable scalability, virtualization, redundancy, and security. These services include load balancing, firewalls and fine grain network metrics. As cloud computing and other large scale virtual server deployments grow, providing these network services in a scalable and manageable fashion is increasingly challenging.
Ideally, each individual network flow in a large scale server deployment is tracked in order to monitor appropriate services, e.g., quality of service, policies, etc., to an identified connection or flow. The network flows are defined as a unidirectional data flow from a single source to a single destination. Protocols have been developed to monitor the characteristics of network flows. NetFlow, developed by Cisco Systems, Inc., and Internet Protocol Flow Information eXport (IPFIX), devised by an Internet Engineering Task Force (IETF) working group, are examples of widely used protocols. In some cases, data center switches may provide the network flow monitoring, but the number of individual flows passing through a single switch may use excessive switch resources to track and manage each flow. In a virtualized network environment there may be thousands of such network flows.